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Abstract 

We propose a neural network model with transient chaos, or a transiently chaotic neural 
network (TCNN) as an approximation method for combinatorial optimization problems, 
by introducing transiently chaotic dynamics into neural networks. Unlike conventional 
neural networks only with point attractors, the proposed neural network has richer and 
more flexible dynamics, so that it can be expected to have higher ability of searching for 
globally optimal or near-optimal solutions. A significant property of this model is that 
the chaotic neurodynamics is temporarily generated for searching and self-organizing, and 
eventually vanishes with autonomous decreasing of a bifurcation parameter corresponding 
to the ” temperature” in usual annealing process. Therefore, the neural network gradually 
approaches, through the transient chaos, to dynamical structure similar to such conven¬ 
tional models as the Hopfield neural network which converges to a stable equilibrium 
point. Since the optimization process of the transiently chaotic neural network is similar 
to simulated annealing, not in a stochastic way but in a deterministically chaotic way, the 
new method is regarded as chaotic simulated annealing (CSA). Fundamental characteris¬ 
tics of the transiently chaotic neurodynamics are numerically investigated with examples 
of a single neuron model and the Traveling Salesman Problem (TSP). Moreover, a main¬ 
tenance scheduling problem for generators in a practical power system is also analysed to 
verify practical efficiency of this new method. 

Keywords- Neural network, Chaos, Transient chaos, Simulated annealing, TSP, Bi¬ 
furcation, Combinatorial optimization problem, Modern heuristics, NP-hard 
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List of Symbols 


Xi : output of neuron i 

Hi : internal state of neuron * 

ujij : connection weight from neuron j to neuron * 

Ii : input bias of neuron * 

a : positive scaling parameter for inputs 

k : damping factor of nerve membrane (0 < k < 1) 

Zi,z : self-feedback connection weight or refractory strength 
e : steepness parameter of output function(e >0) 

E : energy function 

At : time step of the Euler method 
f3 : damping factor for z % and z 

A : the Lyapunov exponent 

Jo : positive parameter 

Gt : generating capacity of unit-* 

h dotal periods for the horizon 

mpi : maintenance periods of unit-* 

Dj : load in period -j 

Rj : reserve margin in period-j 
Ai : set for units belonging to power plant-/ 
hfi hti : specified maintenance periods for unit-* 
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1 Introduction 

A lot of problems in science and technology are related to combinatorial optimization 
problems. Researches in this held aim at developing efficient techniques for finding so¬ 
lutions realizing minimum or maximum values of a objective function subject to a set 
of constraints with many discrete variables. A number of interesting combinatorial op¬ 
timization problems are hard to deal with or intractable because they belong to a class 
of quite difficult problems, the NP (Nondeterministic polynomial) -hard class, which is a 
terminology of computational complexity. That is, to find an exactly optimal solution for 
any NP-hard problem requires a number of computational steps that grows faster than 
any finite power of some appropriate measure of the problem size as long as P / NP. 

To address the NP issue in more details, we briefly give rough explanations of several 
terminologies. The recognition or decision problems which have a known polynomial¬ 
time deterministic algorithm are said to be in class P. On the other hand, the decision 
problems solvable by a polynomial-time nondeterministic algorithm are in class NP, where 
P C NP (Reeves, 1993). If every problem in NP is polynomially reducible to a problem 
Q, Q is NP-hard. Furthermore, if every problem in NP is polynomially transformable 
to a problem Q in NP, Q is NP-complete, which is a class of the hardest problems in 
NP. Therefore, if a polynomial algorithm were to be found for any one of NP-hard or 
NP-complete problems, a polynomial algorithm would be also found for every problem in 
NP, i.e., P = NP which means that all NP decision and NP optimization problems would 
have a polynomial algorithm (Karp, 1972; Garey and Johnson, 1979; Watanabe, 1994). 
A classical and famous example of the NP-hard problem is the optimization version of 
TSP (travelling salesman problem) whose decision version is NP-complete. 

To seek an optimal solution, a lot of exact methods have been so far proposed to deal 
with the combinatorial optimization problems, such as cutting plane methods, dynamic 
programming based on the Bellman’s principle of optimality, branch-and-bound methods, 
and so on. Unfortunately, no exact polynomial algorithm, however, has been found for 
any NP-hard problem up to now. In fact, it is widely believed that P ^ NP though not 
proved. 
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Since all attempts to look for an exact method with polynomial time for NP-hard prob¬ 
lems have failed, many researchers have been paying attention, instead, on approximate 
or heuristic algorithms which seek near-optimal solutions at a reasonable computational 
cost without ensuring optimality or feasibility (Reeves, 1993). ^Frorn the viewpoint of 
the approximate algorithms, the concept of NP-hardness is further partitioned into weak 
NP-hardness and strong NP-hardness (Garey and Johnson, 1979; Ibaraki, 1994b). A 
typical problem with weak NP-hardness is the KNAPSACK problem, which has a fully 
polynomial-time approximation scheme (FPTAS) which gives a near-optimal solution with 
an arbitrary accuracy requirement e(> 0), by polynomial time of 1/e. However, most of 
well-known NP-hard problems are not in the weak NP-hard class. In fact, by the concept 
of Linear-reduction, MAX SNP (strict NP)- complete class and MAX SNP-hard class 
(Papadimitriou and Yanakakis, 1991) which includes many important NP-hard problems, 
can be defined. It has been proved (Arora, Lund, Motwani, and Szegedy, 1992; Ibaraki, 
1994b) that there exists no polynomial-time approximation schemes which can give a so¬ 
lution arbitrarily close to the exact one for MAX SNP-hard class if P ^ NP. Moreover, 
for some NP-hard problems not belonging to MAX SNP-hard class, such as TSP without 
triangle inequality of distances, even an approximate algorithm with guarantee of a finite 
accuracy does not exist as long as P / NP. 

Nevertheless, from the practical viewpoint, approximate algorithms are useful and have 
achieved considerable success when applied to a large number of problems. The approxi¬ 
mate algorithms developed so far include greedy methods, local search methods, random¬ 
ized methods, relaxation techniques, partial enumeration, decomposition and partition 
approaches, and so on (Zanakis, Evans and Vazacopoulos, 1989; Reeves, 1993). Among 
them, the local search methods have been intensively studied, where a concept of PLS 
(polynomial time local search) - complete is also introduced on the basis of PLS- reducibil- 
ity to evaluate the computational complexity (Johnson, Papadimitriou, & Yannakakis, 
1988). Moreover, some modern heuristic or meta heuristic algorithms which are applica¬ 
ble more generally to combinatorial optimization problems rather than problem-specific 
(Reeves, 1993) and which can be related to physical and biological phenomena (Bounds, 
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1987), have recently been developed on the basis of local search; they include simulated 
annealing, tabu search, genetic algorithms and artificial neural networks (Reeves,1993; 
Ibaraki, 1994a). In this paper, we focus on two of such modern heuristics, i.e., simu¬ 
lated annealing and neural networks as well as chaotic dynamics in order to develop a 
new heuristic technique, which we call chaotic simulated annealing for combinatorial op¬ 
timization problems. Since simulated annealing and neural networks are our main topics, 
we give a brief review next. 

The main difficulty in solving combinatorial optimization problems is that straight¬ 
forward algorithms, such as greedy methods and local search methods, tend to become 
trapped in local minima. To avoid getting stuck in local minima, in 1983 Kirkpatrick 
et al. developed simulated annealing with the Monte Carlo technique first proposed by 
Metropolis et al.(1953), which emulates the solid annealing processing by first heating a 
solid to its melting point and then slowly cooling the material at a natural rate related 
to its heat transport speed (Kirkpatrick et al, 1983). Since the optimization processing is 
usually undertaken in a stochastic manner, this annealing mechanics is actually stochastic 
simulated annealing (SSA). Convergence in distribution with the uniform measure over 
the exact solutions, or globally optimal solutions has been theoretically proved by Ge¬ 
nian and Genian as long as the cooling schedule is slow enough to be at least inversely 
proportional to the logarithm of time (Genian & Genian, 1984). A fast annealing with 
generating probability obeying the Cauchy distribution, which cooling scheduling is in¬ 
versely linear in time (Szu & Hartley, 1987), and a very fast annealing with exponential 
cooling schedule (Ingber & Rosen, 1992; Ingber, 1993) have been developed to improve 
the performance of the original SSA (Rosen & Nakano, 1994). 

Up to now, the SSA method has been widely applied to various optimization problems 
with considerable success, but nevertheless it still suffers from several shortcomings. In 
particular, the method requires subtle adjustment of parameters in the annealing schedule 
such as the size of the temperature steps during annealing, the temperature range, the 
number of re-starts and re-direction of the search, etc., due to the intrinsic characteristics 
of its stochastic mechanics (see Kirkpatrick, Gelatt Jr., & Vecchi, 1983; Aarts & Korst, 
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1989; Johnson et ah, 1989; Johnson et al., 1991; Ingber, 1993; Reeves, 1993; Rosen 
& Nakano, 1994). In addition, time consuming due to its Monte Carlo scheme is also 
an annonying problem of SSA. That is, to guarantee convergence to an exact solution, 
SSA would requires even more iterations than complete enumeration for some problems. 
For example, for TSP with n cities, the computation cost of SSA is 0(n n2n ') far more 
than O ( (n — 1)!) of complete enumeration and 0(n 2 2”) of dynamic programming, when 
the logarithmic cooling schedule is used (Aarts & Korst, 1989; Kubo, 1994). Therefore, 
heuristic fast cooling schedules, such as geometric cooling (Johnson, Aragon, McGeoch, 
and Schevon, 1989, 1991), exponential cooling and adaptive cooling (Ingber, 1993; Reeves, 
1993), are applied to SSA to improve its performance. 

On the other hand, as one of met a heuristics, neural network approaches have also 
been shown to be a powerful tool for combinatorial optimization (Hopheld & Tank, 1985; 
Rumelhart, et ah, 1986; Wells, 1992; Peterson & Soderberg, 1993). One of the typical 
models of artificial neural networks is the Hopheld neural networks with symmetric con¬ 
nection weights (Hopheld, 1982, 1984; Hopheld & Tank, 1985). Although these neural 
networks guarantee convergence to a stable equilibrium point due to their gradient descent 
dynamics, the main drawback is that these neural networks suffer from the local minimum 
problems whenever applied to optimization problems. Therefore, various simulated an¬ 
nealing techniques have been combined with neural networks to overcome the weakness. 
A typical example of such neural networks is Boltzmann machines composed of stochastic 
binary neurons with bring probability of the logistic function (Ackey et ah, 1985; Aarts 
& Korst, 1989). Besides, Akiyama et al. (1991) developed Gaussian Machines by adding 
random noises with the Gaussian distribution to an analog neural network, to perform 
SSA processing. On the other hand, instead of stochastic models, several heuristic anneal¬ 
ing approaches which are actually deterministic simulated annealing (DSA), have been 
proposed and applied to neural networks; the approaches include gain sharpening of the 
Hopheld networks (Hopheld & Tank, 1985; Lee & Sheu, 1991; Akiyama et ah, 1991), and 
mean held approximation annealing (Peterson & Anderson, 1987; Hinton, 1989; Geiger & 
Girosi, 1991, Masui & Matsuba, 1991; Peterson & Soderberg, 1993), and so on. 
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While most of the conventional networks fundamentally utilize gradient descent dy¬ 
namics, a number of artificial neural networks with chaotic dynamics have been inves¬ 
tigated towards more complex neurodynamics (e.g., Sompolinsky, Crisanti & Sommers, 
1988; Aihara, Takabe & Toyoda, 1990; Aihara, 1990; Yao & Freeman, 1990; Tani, 1991; 
Inoue & Nagayoshi, 1991; Tsuda, 1992; Nozawa, 1992; Chen, 1993; Adachi et al., 1993; 
Yamada et al., 1993; Nara, Davis & Totsuji, 1993). Among them, we take in this paper 
chaotic neural network (CNN) models which have been proposed by considering graded 
responses, relative refractoriness and spatio-temporal summation of inputs (Aihara et al., 
1990; Aihara, 1990). Unlike the static or near-equilibrium behavior of the conventional 
neural networks, CNN has richer and far-from-equilibrium dynamics with various coexist¬ 
ing attractors, not only of fixed points and periodic points but also of strange attractors 
in spite of the simple equations. Although many reports have approved that the chaotic 
dynamics can be promising techniques for information processing and optimization, the 
convergence problems have not yet been satisfactorily solved in relation to chaotic dy¬ 
namics as far as the authors know. That is, it is usually difficult to decide when to 
terminate the chaotic dynamics, or how to control chaotic behavior for convergence to a 
stable equilibrium point. 

In order to make use of the advantages of both the chaotic neurodynamics and the con¬ 
ventional convergent neurodynamics, this paper aims at developing a transiently chaotic 
neural network (TCNN) for combinatorial optimization problems, by introducing a new 
variable corresponding to the ” temperature ” in usual annealing process into the CNN 
models to harness the chaotic dynamics appropriately. Different from the convergent 
mechanism of the Hopfield neural networks with gradient descent dynamics, the proposed 
model has richer and more flexible dynamics, which can prevent the network from being 
trapped in commonplace local minima, and may obtain acceptably near-optimal solutions 
eventually. A significant property of the new method is that the chaotic neurodynamics 
which is temporarily generated for searching and self-organizing, gradually vanishes with 
autonomous decreasing of a bifurcation parameter, or the ” temperature”, accompanied 
with successive bifurcations from strange attractors into an equilibrium point. In other 
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words, the neural network gradually approaches to dynamical structure similar to the 
Hopheld neural networks, and usually converges to a stable equilibrium solution after van¬ 
ishing of temporary chaotic behavior which we call ” transient chaos” (Chen, 1993). Since 
the optimization process of the transiently chaotic neural network is deterministically 
chaotic rather than stochastic, the proposed approach is regarded as chaotic simulated 
annealing (CSA), in contrast to the conventional stochastic simulated annealing (SSA), 
from the viewpoint of annealing mechanics. There are mainly two significant differences 
between CSA and SSA. The first is that SSA is stochastic on the basis of the Monte 
Carlo scheme while CSA is deterministic with transiently chaotic dynamics. The second 
is that the convergent processing of SSA is undertaken by control of stochastic ’’thermal” 
fluctuations while that of CSA is by control of bifurcation structures. Furthermore, 
the searching space for optimization is also different; namely, SSA fundamentally searches 
all possible states by temporally changing probability distributions achieved by its Monte 
Carlo scheme; on the other hand, the searching of CSA with temporally changing invari¬ 
ant measures determined by its dynamics is restricted to a possibly fractal subspace with 
continuous states. Since the dynamics of TCNN is dissipative, the volume of the searching 
region in CSA is usually very small compared with that of the state space. Therefore, 
CSA can be expected to perform efficient searching because of its reduced search spaces 
if the restriction is adequate to include a global optimum state or its good approximation 
or at least some parts of the basins for such solutions. Since the dynamics of TCNN is 
also deterministic, CSA can be viewed as a method belonging to deterministic simulated 
annealing (DSA). 

To verify and test the new method, transiently chaotic dynamics of a single neuron 
model and networks for the TSP (Traveling Salesman Problem) is numerically analysed 
in this paper. Moreover, a maintenance scheduling problem for generators in a power 
system is also studied to certify the ability of TCNN with CSA as a method of modern 
heuristics for practical combinatorial problems. 

This paper is organized as follows: In the next section, the CNN model is briefly 
described, and then a transiently chaotic neural network as well as the chaotic simulated 
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annealing is proposed for combinatorial optimization problems, and numerically analysed 
to clarify the nonlinear computational dynamics. In section three, TSP with 4 cities, 
10 cities and 48 cities are analysed to investigate efficiency of the model. A large-scale 
combinatorial optimization problem for a practical generator maintenance scheduling of 
a power system is examined in section four. Finally, we give some general remarks to 
conclude this paper. 
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2 Transiently Chaotic Neural Networks and Chaotic Simulated 
Annealing 

In this section, firstly the chaotic neural network (CNN) is briefly summarized. Then, a 
transiently chaotic neural network (TCNN) is introduced. In the last of this section, a 
single neuron model is analysed to demonstrate both transiently chaotic dynamics and 
chaotic simulated annealing (CSA). 

2.1 A model of the chaotic neural network 

The model of the chaotic neural network (CNN) can be described as follows for the purpose 
of this paper (Aihara, Takabe, & Toyoda, 1990): 


Nt) = ! + (!) 

n 

Di{t + 1) = kyi(t) + a(J2 u ijXj(t) + h) - ^(t) (2) 

3 = 1 

(* = 1 ,-,n) 


where, 

Xi : output of neuron i 

Hi : internal state of neuron i 

uiij : connection weight from neuron j to neuron i 

Ii : input bias of neuron i 

a : positive scaling parameter for inputs 

k : damping factor of nerve membrane(0 < k < 1) 

z t : self-feedback connection weight or refractory strength (constant) (z t > 0) 
e : steepness parameter of the output function (e > 0). 

Unlike simple threshold elements in conventional neural networks with gradient descent 

dynamics converging to an equilibrium point, the neuron model of CNN has more complex 

dynamics as a constituent element of neural networks and produces rich spatio-temporal 

dynamics as networks (Aihara, et ah, 1990; Aihara 1990; Adachi et al., 1993; Yamada et 

al., 1993). 

2.2 A model of a transiently chaotic neural network 

As already mentioned, such conventional neural networks as the Hopheld neural networks 
with continuous-time or asynchronously discrete-time state transitions, guarantee conver- 
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gence to a stable equilibrium solution but suffer from local minimum problems. On the 
other hand, although the dynamics of CNN has an intriguing property to move chaotically 
over fractal structure in the phase space, without getting stuck at local minima because 
of accumulation of refractory or self-inhibitory effects (Ailiara et ah, 1990; Aihara, 1990; 
Nozawa, 1992; Adachi et ah, 1993; Yamada, et ah, 1993), the convergence problems of 
the chaotic dynamics have not been satisfactorily solved so far. 

In order to take advantages of both the convergent dynamics and the chaotic dynamics 
but to overcome their disadvantages for combinatorial optimization problems, we propose 
a transiently chaotic neural network (TCNN) by modifying the chaotic neural network, 
as defined below: 


xAt) = -(3) 

lW 1 + e -yi(t)/e V ' 

n 

Ui(t + 1) = kyi(t) + a( J2 VijXj{t) + Ii) — Zi(t)(xi(t) — I 0 ) (4) 

Zi(t + 1) = (1 - f3)zi{t) (5) 

(i = 1 

where, ofy- = uj^uju = 0; £" =1 + h = -dE/dxi 

E : energy function defined in section 3 

Zi(t) : self-feedback connection weight or refractory strength (zi(t) > 0) 

(3 : damping factor of the time-dependent Zi(t) (0 < /3 < 1) 

Iq : positive parameter. 

The difference between CNN and TCNN is the third terms in the right-hand sides of 
eqn.(|) and eqn.({4]) where ZiXi(t ) in eqn.(||) is replaced with Zi(t)(xi(t ) — Iq) in eqn.(fD- 
The term can be related to negative (inhibitory) self-feedback or refractoriness (Aihara 
et al., 1990) with a bias / 0 . We will show later that this neural network has actually 
transiently chaotic dynamics which eventually converges to a stable equilibrium point 
through successive bifurcations like a route of reversed period-doubling bifurcations, with 
the temporal evolution of a new variable Zi(t) according to eqn.(^]). The variable Zi(t) 
corresponds to the temperature in usual stochastic annealing process. Thus, eqn.(|D 
represents an exponential cooling schedule for the annealing. 

In order to compare TCNN with the continuous-time Hopheld neural network (Hop- 
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field, 1984; Hopfield & Tank, 1985), eqns. g-g can be rewritten in forms of differential 
equations as follows: 


dyi 

dt 


X; = 


1 + e~ y d £ 


-k'yi + a'(53 UijXj + h) - Zi(xi - J 0 ) 
j= i 

A = -ffz, 

dt 

(i = 1 ,-,n) 


( 6 ) 

(7) 

( 8 ) 


where, uj 13 = u^ua = 0; £" =1 ^ijXj + h = -dE/dxt. Obviously, eqns.g-g 
are the difference equational version of eqns. g-(g on the basis of the Euler method. 
Comparing eqns.g-g with eqns.g-g, we get the relationships of (1 — A tk') = k, 
A ta! = a, and At/3' = f3, where At is the time step by the Euler method. 

The difference between the proposed model and the continuous-time Hopfield neural 
network is that a nonlinear term —z i (x l — Iq) is added in eqn.g). Since the ’’temperature” 
Zi(t) tends toward zero with time evolution in the form of Zi(t) = Zi(0)e~ /3 ' t , eqns.g-g 
eventually reduce to the continuous-time Hopfield neural network without self-feedback 
connections, namely Wa = 0. From the observation of eqn.g, as the variable Zj can 
be interpreted as the strength of negative (inhibitory) self-feedback connection of each 
neuron, the annealing mechanics is regarded as exponential damping of the 
negative self-feedback connection weights. 

In addition, the relation between the added term and Zi implies that the dynamics of 
this model depend sensitively on z\. In fact, if the value of Zi is fixed, the model of eqns.g- 
(|) is equivalent to CNN with complicated bifurcation structure (Aihara, et ah, 1990; 
Ailiara, 1990) by difference analogue with the Euler method and variable transformation 
(Nozawa, 1994). The damping of Zi produces successive bifurcations in the difference 
version so that the neurodynamics eventually converges from strange attractors to a stable 
equilibrium point as will be shown later. In this paper, for the sake of simplicity, is set 
to decay exponentially as described by eqn.g or eqn.(§), although more skillful damping 
may be required for more efficient optimization processing. 
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2.3 Transiently chaotic dynamics of the single neuron model 

We examine nonlinear dynamics of the single neuron model in this section. 

^From eqns.(0)-(||D, the single neuron model is derived by the Euler method as follows 
(or directly from eqns.(|3])-(|5])): 


X ^ 1 + e-vM/ € 

( 9 ) 

y(t + 1 ) = ky(t) + 7 - z(t)(x(t) - I 0 ) 

( 10 ) 

z(t + 1) = (1 - (3)z(t) 

( 11 ) 


where, 7 = ai*. Obviously, if the value of z(t) is fixed at z 0 , eqns. can be 

easily transformed to the following one-dimensional mapping from y(t) to y(t + 1 ) by 
substituting eqn.([|) into eqn.(fTOj): 


y(t + 1) = ky{t) + 7 ~ M)( x + e _ m/t ~ h) ■ 
The values of the parameters in eqns. 0-0 are set as follows: 


( 12 ) 


k = 0.9; e — 1/250; I 0 = 0.65; ^(0) = 0.08 (13) 

In the following, we vary only [3 and 7 (or a) to investigate the dynamics while other 
parameters are fixed as eqn. (|T3|) . 

Fig.[l] shows the time evolutions of x(t),z(t) and the Lyapunov exponent A of x(t), 
which are calculated by eqns.(PD-(pd|) with (3 = 0 . 001,7 = 0.0 and the initial condition 
?/(0) = 0.5. Here, the Lyapunov exponent A which is generally taken as a crucial index to 
identify orbital instability of deterministic chaos, is defined as follows: 


A = Urn -Y j l n \ dX ^ + l) 

k =0 


nw+oo m ' dx(k ) 


(14) 

and (HJ) for sufficient large 


The Lyapunov exponent is calculated with eqns.(j9| 
iterations m for each fixed value of zq. Namely, positive values of A indicate that dynamics 
of eqn. (|I 2 D , or eqns. (|9|)- (P~ 0 |) actually has orbital instability for the corresponding fixed 
value of Zq. 
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On the other hand, the time evolutions of x(t) and z(t) in Fig.|T] are calculated by 
simultaneous simulation with all of eqns.(|9l)- (|TTl) . Fig.|Ij shows that, with exponential 
damping of z(t), the neuron output x(t) gradually transits from chaotic behavior to a 
fixed point through reversed period-doubling bifurcations; that is, x(t ) behaves erratically 
and unpredictably during the first 300 iterations and eventually converges to a stable fixed 
point around the iteration 950 through e.g. the reversed period-doubling bifurcations. 
Since the Lyapunov exponents in the first 300 iterations are mostly positive, the behavior is 
understood as chaotic in most of the region with possible periodic windows. Fig.|l] indicates 
that chaotic fluctuations decrease with the damping of z(t ) and eventually vanish. In 
other words, it is a kind of transiently chaotic dynamics, and the dynamical structure of 
the neural network almost coincides with the Hopfield network, when the value of z(t) 
decreases small enough. 

Furthermore, we vary the values of the parameters f3 and 7 , to examine their influences 
on the neural dynamics. Fig.^(a) and Fig.[|(b) are the time evolutions of x[t ) and z(t), 
when f3 = 0.002 and f3 = 0.0008, respectively. In Fig.[2|(a), the chaotic dynamics of x(t ) 
vanishes quickly because z(t) decreases rapidly owing to the larger value of (3 . On the 
other hand, in Fig.^(b), the chaotic dynamics of x(t) in turn lasts longer due to the small 
value of (3. Therefore, it is evident that (3 governs the bifurcation speed of the transient 
chaos. 

The convergence procedure of the transiently chaotic neural network is fully deter¬ 
ministic, namely, starting from deterministically chaotic dynamics, through a reversed 
period-doubling route with decreasing the value of z which corresponds to the temper¬ 
ature of usual annealing, finally reaching a stable equilibrium solution. The mechanics 
of TCNN is called chaotic simulated annealing (CSA) which is regarded as a kind of de¬ 
terministic simulated annealing (DSA). Moreover, compared with the stochastic thermal 
fluctuations of SSA, the convergence process of CSA is characterized as the nonlinear 
bifurcations which drive the neurodynamics to converge from strange attractors to an 
attracting fixed point. 
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3 Application to the Traveling Salesman Problem 

A classical and famous combinatorial optimization problem which is NP-hard is the trav¬ 
elling salesman problem (TSP) which seeks the shortest route through n cities, visiting 
each once and only once and returning to the starting point. TSP can be generally divided 
into symmetric TSP with d^ = dji , and asymmetric TSP with dij ^ dji, where dij is the 
distance from city-i to city-j. For symmetric TSP, there are (n — l)!/2 possible tours 
(or solutions), while there are (n — 1)! possible tours for asymmetric TSP. Obviously, the 
number of possible tours grows faster than any finite power of n. As opposed to intu¬ 
ition, asymmetric TSP is actually easier to solve than symmetric TSP in cases such that 
distances are randomly generated(Kubo,1994). Moreover, to categorize TSP further, it 
is important whether the TSP satisfies triangle inequality of distances, dik + dkj > d l} 
or not. If a symmetric TSP satisfies the triangle inequality, there exists an approximate 
algorithm which can give a route whose length is less than 3/2 times of that of the exact 
solution (Christofides, 1976; Garey & Johnson, 1979; Kubo, 1994). A typical example 
of the symmetric TSP satisfying the triangle inequality is TSP on the plane under the 
Euclidean metric. 

TSP is simple to state but difficult to solve exactly due to the NP-hardness. For exam¬ 
ple, at a reasonable computational cost, it is thought that both usual integer programming 
and dynamic programming can deal only with 20 ~ 30 city TSP (Ibaraki, 1994a). Re¬ 
cently, it was reported that 2392 and 4461 city symmetric TSP had been solved by branch 
and cut methods (Padberg, & Rinaldi, 1991; Ibaraki, 1994a). Compared with symmet¬ 
ric TSP, as mentioned above, asymmetric TSP is easier to deal with. In fact, a 500000 
city asymmetric TSP has been solved by the branch-and-bound method (Miller & Pekny, 
1991). 

On the other hand, to find acceptably near-optimal solutions of TSP, a large number of 
approximate algorithms have been developed, such as, greedy methods, nearest neighbor 
methods, insertion methods, the Christofides’ algorithm, space filling curve methods, the 
Lin-Kernighan algorithm, 2-OPT, 3-OPT and so on (Kubo, 1994). Among them, the 
Lin-Kernighan algorithm (Lin & Kernighan, 1973) is well known to be very efficient in 
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both the speed and the quality of solutions found; it is reported that solutions within 2% 
of the optimum for symmetric TSP on the plane with as many as 50000 cities are found 
by the Lin-Kernighan algorithm (Johnson, 1987; Johnson et ah, 1988). 

Since Hopfield and Tank (1985) applied their neural networks to TSP, TSP has been 
intensively studied in the field of artificial neurocomputing, ft was recently reported 
(Nozawa, 1992; Yamada et ah, 1993; Nozawa, 1994) that CNN or equivalent models 
may have efficient searching ability for such combinatorial optimization problems as TSP 
because they can escape from local minima due to accumulated negative self-feedback 
inputs or refractory effects (Aihara et ah, 1990). This searching ability of CNN, however, 
is a two-edged sword because CNN frequently escapes even from globally optimal or near- 
optimal states. Therefore, it is an important problem to make CNN converge to good 
solutions while utilizing the searching ability transiently. This is the problem to which 
TCNN with CSA may be useful. 

In this paper, symmetric TSP on the plane is used to test the ability of TCNN with 
CSA as a heuristic method for combinatorial optimization. We adopt the formulation for 
TSP by Hopheld and Tank (1985). Namely, a solution of TSP with n cities is represented 
by a n x n - permutation matrix, where each element corresponds to output of a neuron 
in a network with n x n lattice structure. Assume Xij to be the neuron output which 
represents to visit city-i in visiting order-j. A computational energy function which is to 
minimize the total tour length while simultaneously satisfying all constraints takes the 
following form: 


Ti7 n n n n 

z i= i j= i j= i i= i 

n n n 

+^EEE (%kj+i T 3'kj—i)3'ijdik (15) 

z i=ij=ik=i 

where x i0 = x in and x in+ i = x^. W\ and W 2 are the coupling parameters corresponding 
to the constraints and the cost function of the tour length, respectively, where dij is the 
distance between city-i and city-.)• 

By setting each connection weight as same as the Hopheld neural network, the 
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difference equations describing the network dynamics of TCNN for the TSP are obtained 
from eqns.(^|)-(|]) as follows: 


1 _|_ e -Vij(t)/e 
n n 

Uij(t + 1) = kyij{t) - z(t)(xij(t) - I 0 ) + (^{-Wxi^Xiiit) + ^2x kj (t)) 

lAj k ^i 

n n 

M> ^2 djk x ki +1 (t) + djkXki-i(t)) + W\} 

k^i k^i 

(i,j = l,...,n) 

z{t + 1) — (1 — ff)z(t) 


(16) 


(17) 


(18) 


where the variable z(t) is assumed to be common to all the neurons, and = W\. 

In order to improve the computation performance, we code a discrete neuron output 
xfj instead of the continuous output Xij (■ i,j = 1, as follows: 


x D {t) = 1 1 ift (*)/(« X «) 


(19) 


0 otherwise 

Since the output x t] takes a continuous value between 0 and 1, the corresponding 
energy function of eqn. (P~5|) is also continuous. We call the energy function of eqn.([T5D 
with the continuous output x %] and that with the discrete output xfj as the Continuous 

Energy Function E c and the Discrete Energy Function E D , respectively. 


In the following simulations, W\ = Wi = 1 and other parameters are set as eqn. (|l3D 
except a and /3, and states of neurons are cyclically updated. 


3.1 Experiments on TSP with 4 cities 

Fig|| shows the Hopfield-Tank original data on TSP with 10 cities (Wilson & Pawley, 
1988). In this subsection, we use only the first 4 cities to examine fundamental charac¬ 
teristics of TCNN when applied to TSP. 

Fig.|] shows the time evolutions of the continuous and discrete energy functions and z, 
when a = 0.015 and /3 = 0.001. Fig]5] illustrates the time evolution of each neuron output 
x^, while Fig.|]is amplification of in. 
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Figs.Q-j^ shows that the time evolution of the TCNN changes from chaotic behavior 
with larger fluctuations at the early stage, through periodic behavior with the period 5 and 
chaotic behavior with smaller fluctuations, into the later convergent stage. After about 
iteration 1300, when z becomes so small that the convergent characteristic dominates the 
dynamics, the neural network state finally converges to a fixed point corresponding to a 
global minimum (tour length = 1.18). In Figs.[|-|(| and the subsequent simulations, one 
iteration means one cyclic updating of all neuron states. The output patterns in Fig.^| 
indicates that the best route is 3-4-1-2-3. In addition, it is found that other representations 
of best routes with the same energy value (tour length = 1.18) such as 2-3-4-1-2 and 4-1- 
2-3-4 can be obtained, depending on the initial conditions of . 

The simulation results imply that, TCNN firstly reaches a valley with noninteger solu¬ 
tions probably involving a part of the basin of a certain global minimum after z becomes 
small enough for disappearance of transient chaos, and then TCNN converges to the 
global minimum with an integer solution starting from the state in the valley with further 
decreasing of z. For instance, according to Figs|§-|(| at about iteration 1100, the system 
reaches a valley with a noninteger solution after the transiently chaotic behavior, and then 
rapidly transits from the noninteger solution through falls toward a nearly integer solution 
which corresponds to the global minimum. Consequently, the dynamics of the TCNN can 
be classified into two phases, depending on the value of z, i.e., (1) the ’’chaotic searching 
phase” with transiently chaotic dynamics for a relatively large z, and (2) the ’’stable 
convergent phase” with convergent dynamics similar to the Hopfield neural network for 
a relatively small z. The chaotic searching phase seems to act as searching for a valley 
belonging to the basin of a certain global minimum, while the stable convergent phase, 
starting from the valley, is to press the neural network to converge to the corresponding 
integer solution of the global minimum. For the case of Figs.|]-|6|, the ’’chaotic searching 
phase” is until about iteration 1100, and the ’’stable convergent phase” starts at about 
iteration 1100 when z is small enough. 

Moreover, it is also found that the period-5 behavior in Figs.^-j^ is actually incomplete 
traveling waves (Wells, 1992), where the solutions correspond to local minima. 
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Figs.f^ show the time evolutions of the continuous energy function when a and (3 
are largely varied. For the cases of Fig.[7](b) and Fig.[|(b) where (3 and a is extremely 
large respectively, the chaotic dynamics vanishes too quick to realize sufficient chaotic 
searching. Therefore, the solution rapidly converges to a local minimum like the Hopheld 
neural network. In contrast, for the case of Fig.[7](a) where (3 is very small, the strong 
chaotic dynamics lasts too long to converge within the prescribed iterations. On the other 
hand, for the case of Fig.|8](a) where a is extremely small so that the energy function can 
not sufficiently affect the neurodynamics, the network converges to a wrong solution. The 
obtained solutions in these cases are either infeasible solutions or local minima. 

It should be noticed that a represents the influence of the energy function to the 
neuro dynamics, or the balance between the self-feedback or refractory term inducing the 
chaotic dynamics and the gradient term dE/dxij inducing the convergent dynamics. If 
a is too large, the influence of the energy function becomes too strong to generate the 
transient chaos. On the other hand, the energy function cannot be sufficiently reflected 
in the neurodynamics if a is too small. 

3.2 Solution for TSP with 10 cities and 48 cities 

As a next example, we have analysed 10 city TSP of the data in Fig|| with a = 0.015, 
f3 = 0.015, 0.010, 0.005, 0.003, and with 5000 different initial conditions of yij generated 
randomly in the region [-1,1] for each value of f3. The results are summarized in Table [Tj. 

As shown in Table [Lj. TCNN converges averagely with 81 iterations (for discrete energy 
function) when (3 = 0.015, where among 5000 cases, 31 cases are to local minima (tour 
length = 2.74,2.78,...), 23 cases are to infeasible solutions and 4946 cases are to global 
minima (Tour length=2.70). That is, the rate of global optimization is 98.9% for (3 = 
0.015. The best route is shown in Fig.||. As mentioned previously, the different initial 
conditions of yij can derive different output representations of global optimal solutions 
with the same tour length. 

The parameter f3 can be considered as a damping speed parameter of the negative 
self-feedback strength or the ”temperature” z, which controls the annealing schedule. In 
order to produce stronger and longer chaotic dynamics, we set (3 small as (3 = 0.005 
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and 0.003, so that the temperature z decreases more slowly. According to Table [I|, the 
results are considerably improved, e.g., when 0 = 0.005, TCNN converges averagely 
with 234 iterations and all the obtained solutions are feasible. Besides, the rate of global 
optimization reaches 99.9%, which is extremely high compared with the rate less than 37% 
obtained by the Hopheld neural network (Akiyama, et ah, 1991; Nozawa, 1992, 1994). 
Finally as shown in Table [I], 100% of the rate of global optimization is also realized by 
simulations with 0 = 0.003 for this toy problem with 10 cities. 

On the other hand, a kind of SSA with the Metropolis algorithm is also applied to the 
same 10 city TSP to compare with CSA in terms of computational cost. In this calculation, 
SSA adopts the same cooling schedule as that of CSA, namely the exponential cooling 
schedule of eqn.(|]) with 0 = 0.00004 for annealing. The results of SSA for this 10 city 
TSP with 100 different initial conditions are summarized as follows: 

Rate of global minima : 92 % 

Average iterations : 58700 

Here, one iteration means one cyclic updating of all neuron states according to the 
Metropolis criterion and the temperature is decreased by one step after each iteration. 

Obviously, CSA is much faster and more efficient than this SSA for the 10 city TSP. 

Next, we adopt the 48 U.S.A. city TSP (Reinelt, 1994) as another example for CSA; 
the parameters are set as follows: 

k = 0.9; e = 1/250; J 0 = 0.5; *(0) = 0.10; a = 0.015,0 = 5.0 x 10" 5 ; H0 = 1 ;W 2 = 1/3 

Fig.^j and Fig.[TI] are the optimal route (the route length = 10628) and a near-optimal 
route (the route length = 10805) in the 48 U.S.A. city TSP, respectively. The results with 
100 different initial conditions are summarize in Table |2|, where the average iterations is 
25632. The results show that 84% solutions converge to near-optimal routes like Fig.pR| 
while 5% solutions converge to the optimal route of Fig.|9] from the 100 random initial 
conditions. 

The calculations in this paper are based on cyclic updating of neuron states. There¬ 
fore, as mentioned before, one iteration means that all neuron states are cyclically updated 
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once. On the other hand, we have obtained similar results with a randomly asynchronous 
updating scheme of TCNN for the combinatorial optimization problems (Chen & Aihara, 
1993, 1994a, 1994b), where the difference between cyclic updating and randomly asyn¬ 
chronous updating is only that the periodic orbits due to the cyclic updating disappeared 
in the chaotic searching phase when the randomly asynchronous updating scheme was 
adopted. 
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4 Application to Maintenance Scheduling for Generators in a 
Power System 

Every generating unit in power systems should be maintained once during the specihed 
horizon such as during one year or two years. The maintenance scheduling for generators 
is to level the reliability risks with respect to the load by properly maintaining each 
generating unit throughout the horizon. Generally the reserve margin which is defined by 
the rate of generating capacity reserve versus load as shown in Fig.|ll], is widely used as 
a risk index (Chen & Toyoda, 1991). In Fig.|TT], the vertical and horizontal lines for each 
unit block indicate its generation capacity and maintenance period, respectively. Since 
generating units have different sizes of capacities and different maintenance periods, to 
level the reserve margins throughout the horizon becomes a large-scale combinatorial 
optimization problem. Besides, there are two practical constraints which should be taken 
into consideration; the one is that each unit has to be maintained during its specihed 
periods, and the other is that units belonging to the same power plant should not be 
maintained simultaneously in the same period. 

4.1 Formulation of maintenance scheduling for generators 

In order to level the reserve margins, the maintenance scheduling for generators can be 
formulated as the following 0-1 integer programming (Chen, 1993): 

Max Min R, = — ^ ~ ^ k =Mm Pi +i x ik ) ~ £± ( 20 ) 

Xij j=l,...,h 7 Dj 


subject to 


Y,kihL x ik = 1 (i = l,...,n) 

(Xt* i £ Ai X ik)(J2m£AiJ2k=j- mP m+l x mk)=0 

k=j—mpi ~\-1 rrv=f^i 

(i = 1 


( 21 ) 


( 22 ) 


where, x t j represents whether the unit-i is maintained for mpi periods from period-j 
or not, i.e., 



1 if unit — i is maintained for rrip t periods from period — j 
0 otherwise 
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Gi\ generating capacity of unit-i, n: total number of units, Atotal periods for the 
horizon, mpf maintenance periods of uiiit-f, Df load in period-j, Rj\ reserve margin in 
period- 7 , hf t hti. specified maintenance periods for unit-f, Af. set for units belonging 
to power plant-h 


Eqn.(21) represents the constraint that each unit should be maintained once and only 
once during its specified periods. Eqn.(^) is the constraint which represents that the 
units belonging to the same power plant should not be maintained simultaneously in the 
same period. 

In the left-hand side of eqn.(|22|), the Erst term calculates whether the unit-i is main¬ 
tained in period-j or not, namely, the first term is larger than zero if maintaining, other¬ 
wise this term is zero. The second term checks whether the other units belonging to the 
same power plant as unit-i are maintained in the same period or not. 

^From the definition of Xij, (hfi — hti + 1) neurons are necessary for one unit. Therefore, 
the total number of neurons is YGi= \ {hfi ~ hti + 1 ) for the neural network computing the 
maintenance scheduling. 


According to Eqns.(|20| ) - (|22|) , the energy function E for minimization is constituted as 
follows: 


EE 

i=i j =i 


h n rir hti 

£ = E-V? + E^(!- E ^) 2 + 

3=1 »=1 k=hfi 

2 J J 


n h Wr 


E Si*)! E E %mk) 


(23) 


ieAi 

k=j-mpi +1 


meAi k=j-mp m +1 
m^i 


where, the first term in the right-hand side is weighted summation for the total re¬ 
serve margins; the second and third terms correspond to the constraints; W±, W 2 are the 
coupling strengths corresponding to the constraints; A j is a positive weighting parameter. 


It is evident that minimizing eqn. (|23|) is not for leveling the reserve margins, but for 
maximizing the weighted summation of the total reserve margins meanwhile satisfying the 
constraints. Therefore, in order to level the reserve margins approximately, the weighted 
parameters A j should be set appropriately for the concerned problems (Chen, 1993). 
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Substituting —dE/dxij into eqns.@-(^D, we obtain the transiently chaotic neural net¬ 
work for the maintenance scheduling of generators. 

4.2 Simulation of maintenance scheduling 

We apply the transiently chaotic neural network to the maintenance scheduling for gen¬ 
erators in a practical power system with 117 units belonging 40 power plants (Chen & 
Aihara, 1993). The horizon (one year) is divided into 26 periods (1 period is 2-week); 
Xj = Wl = W2 = 1; the total number of neurons is 2384. 

All the solutions obtained by simulations with 100 random initial conditions are feasible 
and the minimal reserve margins among 26 periods have the same value of 24.2%. It has 
been observed that the solutions largely depend on the setting of the weighting parameter 
Xj compared with other parameters W\ and W 2 ■ The average CPU time for one solution 
is approximately 26 seconds with FACOM M-770 (22MIPS). 

5 Discussions 

To cope with the combinatorial optimization problems, we have proposed the chaotic simu¬ 
lated annealing with the transiently chaotic neural network, as an approximation method. 
The transiently chaotic dynamics is utilized for global searching and self-organizing where 
accumulation of refractory or self-inhibitory effects of chaotic neuron models prevents the 
network from getting stuck at local minima. After the transiently chaotic dynamics has 
vanished, the proposed neural network then is fundamentally reined by the gradient de¬ 
scent dynamics and usually converges to a stable equilibrium point like the Hopfield neural 
network. The features of the present method are summarized as follows: 

1. The dynamics of the transiently chaotic neural network is characterized as transient 
chaos. 

2. The proposed model gradually approaches, through the transient chaos, to dynam¬ 
ical structure similar to the Hopfield neural network which converges to a stable 
equilibrium point. 
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3. In contrast to the conventional stochastic simulated annealing , the mechanics of the 
proposed method is regarded as chaotic simulated annealing. 

4. Routes of optimization by the chaotic simulated annealing are through successive 
bifurcations from strange attractors to an equilibrium point. 

5. The proposed annealing mechanics is realized as damping of negative (inhibitory) 
self-feedback connection weights. 

The main difference between SSA and CSA is that the annealing process of CSA is asso¬ 
ciated with a series of the bifurcations, rather than decreasing statistic fluctuations based 
on the Monte Carlo scheme in SSA. Since the mechanics of CSA is based on autonomous 
searching in a reduced possibly fractal subspace by simple deterministic dynamics, CSA 
can be expected to require just short computation time. The characteristics of CSA and 
SSA are summarized in Table |3j. 

This method of CSA provides a new mechanism of deterministic simulated annealing 
(DSA), which is different from other methods of DSA, such as gain sharpening of the 
Hopfield network (Hopfield & Tank, 1985; Lee & Sheu, 1991; Akiyama et al., 1991) and 
mean field approximation annealing (Peterson & Anderson, 1987; Hinton, 1989; Geiger & 
Girosi, 1991, Masui & Matsuba, 1991; Peterson & Soderberg, 1993). 

Numerical experiments of TCNN on TSP and the maintenance scheduling problem 
for generators have shown that the present approach has high efficiency to converge to 
globally optimal or near-optimal solutions on these small-scale problems of combinatorial 
optimization. When compared with SSA, CSA has a peculiar characteristic that the 
searching region in the state space is restricted to a small possibly fractal structure whose 
volume should be zero with respect to the Lebesgue measure of the whole state space. 
The possible reason why CSA has the high ability of global search in spite of its small 
searching region is that the deterministic dynamics due to mutual interactions among 
neurons, which reflect the problem structure such as costs and constraints, restricts the 
searching region efficiently so that the fractal region includes at least some parts of the 
basins associated with globally optimal or near-optimal solutions. Besides,to attain faster 
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speed, hardware implementations of CSA (Horio et al.,1994) is also an important subject. 

One may ask what is difference between stochastic noises and chaotic noises if the 
stochastic noises are generated by a pseudo-random number generator in digital com¬ 
puters. There are several aspects on this problem. Firstly, the usual chaotic noises are 
different from the conventional pseudo-random noises produced by say the linear congru¬ 
ence method because the former own short-term autocorrelation peculiar to the chaotic 
dynamics while the latter are nearly uncorrelated (Hayakawa & Sawada, 1994). In other 
words, although the pseudo-random noises are generated by a kind of chaotic dynam¬ 
ics, they are particularly designed to be almost uncorrelated due to the largely positive 
Lyapunov exponent with homogeneous expansion and coarse-graining to integers. It is 
suggested by comparing effects of three kind of additive noises to gradient descent dy¬ 
namics for optimization, namely pseudo-random noises, chaotic noises by the logistic map 
(May, 1976) and their randomly shuffled version, that the short-term correlation of the 
logistic chaos is useful for optimization (Hayakawa & Sawada, 1994). Another aspect is 
mutual interaction among chaotic elements realized in TCNN. It is reported by globally 
coupled maps (Kanako, 1992) that coupled chaotic dynamics can generate inherent char¬ 
acteristics like breakdown of the law of large numbers and coherent irregularity (Tomita, 
1984), which cannot be produced by just coupling of random noises. 

Finally, we should stress that, although the proposed approach to combinatorial opti¬ 
mization problems by transiently chaotic neural networks with chaotic simulated anneal¬ 
ing provides a new kind of approximation methods, further studies, in particular, strict 
comparison with the excellent techniques already in use (Johnson, 1987) are indispens¬ 
able for the evaluation of the true ability. In fact, it seems that the TCNN with CSA can 
not compete with a heuristics special for a certain problem, such as the Lin-Kernighan 
algorithm for TSP. As a not problem-specific but general method, however, CSA may be 
worth examing the ability for combinatorial optimization. 

It should be also noted, on the other hand, that although the number of cities of the 
TSP analysed in this paper is so small as 10 and 48, the dimension of the corresponding 
chaotic mapping are 100 and 2304 which numbers are fairly large as the dimension of 
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chaotic dynamical systems so far studied. The analyses on the nonlinear dynamics of 
such large-scale chaotic systems and its relationships with such information processing as 
combinatorial optimization are important and interesting future problems. 
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Captions of Figure and Table 


• Figure [T] : Time evolutions of x(t),z(t) and the Lyapunov exponent A in the single 
neuron dynamics. x(t): the output of the neuron, z(t): the self-feedback connec¬ 
tion weight, or the damping variable corresponding to the temperature in annealing 
process. 

• Figure |2| : Time evolutions of the single neuron model with different values of (3. (a) 
/3 = 0.002,7 = 0.0, (b) (3 = 0.0008,7 = 0.0. 

• Figure : The Hopheld-Tank original data of TSP with 10 cities. 

• Figure : Time evolutions of continuous and discrete energy functions and z(t) in 
simulation of TCNN for TSP with 4 cities. f3 = 0.001, a = 0.015. 


• Figure [5] : Time evolution of each neuron output for TSP with 4 cities, a = 
0.015, (3 — 0.001. 

• Figure ^ : Amplification of x\\ in Fig.5. 

• Figure [7] : Time evolutions of the continuous energy function with different values of 
j3. (a) (3 = 0.0001, a = 0.015, (b) f3 = 0.1, a = 0.015. 


• Figure^ : Time evolutions of the continuous energy function with different values of 
a. (a) [3 = 0.001, a = 0.001, (b) (3 = 0.001, a = 0.1. 

• Figure || : The optimal route for the 48 U.S.A. City TSP 


Figure [TO] : A near-optimal route for the 48 U.S.A. City TSP 


• Figure 0: Maintenance scheduling for generators in a power system. 


• Table [l] : Results of 5000 different initial conditions for each value of [3 on the 10 
city TSP by CSA. 


Table |] : Results of 100 different initial conditions on the 48 U.S.A. city TSP by 
CSA. 
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• Table ^ : Comparisons between the chaotic simulated annealing (CSA) and the 
conventional stochastic simulated annealing (SSA) in neural networks. 
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(P = 0.001; 7 = 0.0) 


Figure 1: Time evolutions of x(t), z(t ) and the Lyapunov exponent A in the single neuron dynamics 
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(a) (j3 = 0 . 002 ; 7 = 0 . 0 ) 


(b) (/3 = 0.0008; 7 = 0.0) 


Figure 2: Time evolutions of the single neuron model with different values of (3 
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Figure 3: The Hopfield-Tank original data of TSP with 10 cities 
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(13 = 0.001; a = 0.015) 

Figure 4: Time evolutions of continuous and discrete energy functions and z(t) in simulation of TCNN 
for TSP with 4 cities 
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{(i = 0.001; a = 0.015) 

Figure 5: Time evolution of each neuron output Xi 


for TSP with 4 cities 
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(/3 = 0.001; a = 0.015) 


Figure 6: Amplification of Xn in Fig.5 
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(a) (/3 — 0.0001; a — 0.015) 


(b) (/3 — 0.1; a — 0.015) 


Figure 7: Time evolutions of the continuous energy function with different values of /3 
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(a) (P — 0.001; a — 0.001) 


(b) (p — 0.001; a — 0.1) 


Figure 8: Time evolutions of the continuous energy function with different values of a 
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(Tour Length = 10628) 

Figure 9: The optimal route for the 48 U.S.A. city TSP 
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(Tour Length = 10805) 

Figure 10: A near-optimal route for the 48 U.S.A. city TSP 
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Figure 11: Maintenance scheduling for generators in a power system 
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Table 1: Results of 5000 different initial conditions for each value of /3 on the 10 city TSP by CSA 


(ct = 0.015) 

P = 0.015 

p = 0.010 

P = 0.005 

P = 0.003 

Rate of global minima (%) 

4946 (98.9%) 

4969 (99.4%) 

4998 (99.9%) 

5000 (100%) 

Rate of local minima (%) 

31 (0.6%) 

13 (0.3%) 

2 (0.1%) 

0 (0.0%) 

Rate of infeasible solutions (%) 

23 (0.5%) 

18 (0.3%) 

0 (0.0%) 

0 (0.0%) 

Average iterations 
for convergence 

81 

119 

234 

398 
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Table 2: Results of 100 different initial conditions on the 48 U.S.A. city TSP by CSA 


Route length 

Number (%) 

Error % 

*10628 

5 (5%) 

0 % 

10805 

84 (84%) 

1.665% 

10937 

3 (3%) 

2.907% 

10992 

3 (3%) 

3.425% 

others 

5 (5%) 



*10628 : the best length 
Error : error to the best length 
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Table 3: Comparisons between the chaotic simulated annealing (CSA) and the conventional stochastic 
simulated annealing (SSA) in neural networks 



CSA 

SSA 

Searching mechanics 

deterministic chaos 

stochastic transition 

Annealing and Convergence 

bifurcations to a near- 
optimal equilibrium point 
by decreasing self-connection 

realization of a near- 
optimal ground state 
with decreasing temperature 

Global searching space 

fractal subspace 

all possible states 

Local searching 
direction 

uniquely 

determined by mutual 
interactions among neurons 

probabilistically 
determined by mutual 
interactions among neurons 

















